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Abstract 



We obtain new outer bounds on the capacity regions of the two-user multiple access 
channel with generalized feedback (MAC-GF) and the two-user interference channel 
with generalized feedback (IC-GF). These outer bounds are based on the idea of de- 
c/3 \ pendence balance which was proposed by Hekstra and Willems [1]. To illustrate the 
^ ' usefulness of our outer bounds, we investigate three different channel models. 

\ We first consider a Gaussian MAC with noisy feedback (MAC-NF), where transmit- 

i ter k, k = 1, 2, receives a feedback Ypj,, which is the channel output Y corrupted with 

' additive white Gaussian noise Zk. As the feedback noise variances (j| , /c = 1,2, be- 

OO ■ * 

come large, one would expect the feedback to become useless. This fact is not reflected 

^ . by the cut-set outer bound. We demonstrate that our outer bound improves upon the 

cut-set bound for all non-zero values of the feedback noise variances. Moreover, in the 
OO . „ 

O ' limit as cri oo, k = 1,2, our outer bound collapses to the capacity region of the 

Gaussian MAC without feedback. 

•rH , 

^ . Secondly, we investigate a Gaussian MAC with user-cooperation (MAC-UC), where 

^ . each transmitter receives an additive white Gaussian noise corrupted version of the 

channel input of the other transmitter [2]. For this channel model, the cut-set bound is 
sensitive to the cooperation noises, but not sensitive enough. For all non-zero values of 
cooperation noise variances, our outer bound strictly improves upon the cut-set outer 
bound. Moreover, as the cooperation noises become large, our outer bound collapses 
to the capacity region of the Gaussian MAC without cooperation. 

Thirdly, we investigate a Gaussian IC with user-cooperation (IC-UC). For this 
channel model, the cut-set bound is again sensitive to cooperation noise variances as in 
the case of MAC-UC channel model, but not sensitive enough. We demonstrate that 
our outer bound strictly improves upon the cut-set bound for all non-zero values of 
cooperation noise variances. 



*This work was supported by NSF Grants CCF 04-47613, CCF 05-14846, CNS 07-16311 and CCF 07- 
29127. 



1 Introduction 



It is well known that noiseless feedback can increase the capacity region of the discrete 
memoryless multiple access channel as was shown by Gaarder and Wolf in [3] . The multiple 
access channel with generalized feedback (MAC-GF) was first introduced by Carleial [4]. The 
model therein allows for different feedback signals at the two transmitters. For this channel 
model, Carleial [4] obtained an achievable rate region using block Markov superposition 
encoding and windowed decoding. An improvement over this achievable rate region was 
obtained by Willems et. al. in [5] by using block Markov superposition encoding combined 
with backwards decoding. 

Inspired from the uplink MAC-GF channel model, the interference channel with general- 
ized feedback (IC-GF) was studied in [6], [7], (also seethe references therein) where achievable 
rate regions were obtained. It was shown in [6] and [7] that for the Gaussian interference 
channel with user cooperation (IC-UC), the overheard information at the transmitters has a 
dual effect of enabling cooperation and mitigating interference, thereby providing improved 
achievable rates compared to the best known evaluation of the Han-Kobayashi achievable 
rate region [8] , [9] . 

As far as the converses are concerned for the MAC-GF and the IC-GF, a well known outer 
bound is the cut-set outer bound. The cut-set bound allows all input distributions, thereby 
permitting arbitrary correlation between the channel inputs and hence is seemingly loose. 
The idea of dependence balance was first introduced by Hekstra and Willems [1] to obtain 
outer bounds on the capacity region of single output two-way channel. In contrast to the cut- 
set bound, the dependence balance bound provides an additional non-trivial restriction over 
the set of allowable input distributions thus leading to a potentially tighter outer bound. 
In the same paper [1], the authors give a variant of this bound for the two- user discrete 
memoryless MAC with noiseless feedback from the receiver. 

In this paper, we use the idea of dependence balance to obtain new outer bounds on 
the capacity regions of the MAC-GF and the IC-GF. To show the usefulness of our outer 
bounds, we will consider three different channel models. 

We first consider the Gaussian MAC with different noisy feedback signals at the two 
transmitters. Specifically, transmitter k, k = 1, 2, receives a feedback Yp^ = Y + Zk, where Y 
is the received signal and Z^. is zero-mean, Gaussian random variable with variance (t|^ . The 
capacity region is only known when feedback is noiseless, i.e., Yp^ = Yp^ = Y, in which case 
the feedback capacity region equals the cut-set outer bound, as was shown by Ozarow [10]. 
For the case of noisy feedback in consideration, the cut-set outer bound is insensitive to 
the noise in feedback finks, i.e., it is not sensitive to the variances of Zi and Z2. As the 
feedback becomes more corrupted, or in other words, as (t|^ , a^^ become large, one would 
expect the feedback to become useless. This fact is not accounted for by the cut-set bound. 
We show that our outer bound strictly improves upon the cut-set bound for all non-zero 
values of {o-%_^,a\^). Furthermore, as {cr'^zv^'z'^ become large, our outer bound collapses to 
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the capacity region of Gaussian MAC without feedback, thereby estabhshing the feedback 
capacity region. We should mention here that applying the idea of dependence balance 
to obtain improved outer bounds for Gaussian MAC with noisy feedback was proposed by 
Gastpar and Kramer in [11]. 

Secondly, we investigate the Gaussian MAC with transmitter cooperation. Sendonaris, 
Erkip and Aazhang [2] studied a model where each transmitter receives a version of the 
other transmitter's current channel input corrupted with additive white Gaussian noise. 
They named this model as user cooperation model. This model is particularly suitable for 
a wireless setting since the transmitters can potentially overhear each other. An achievable 
rate region for the user cooperation model was given in [2] using the result of [5] and was 
shown to strictly exceed the rate region if the transmitters ignore the overheard signals. 

We evaluate our outer bound for the user cooperation setting described above. In contrast 
to the case of noisy feedback, the cut-set bound for the user cooperation model is sensitive 
to cooperation noise variances, but not too sensitive. Intuitively speaking, as the backward 
noise variances become large, one would expect the cut-set bound to collapse to the capacity 
region of the MAC without feedback. Instead, the cut-set bound converges to the capacity 
region of the Gaussian MAC with noiseless output feedback [10]. On the other hand, in the 
limit when cooperation noise variances become too large, our bound converges to the capacity 
region of the Gaussian MAC with no cooperation, thereby yielding a capacity result. For all 
non-zero and finite values of cooperation noise variances, our outer bound strictly improves 
upon the cut-set outer bound. Our dependence balance based outer bound coincides with 
the cut-set bound only when the backward noise variance is identically zero and both outer 
bounds collapse to the total cooperation hue. 

Thirdly, we evaluate our outer bound for the Gaussian IC with user cooperation (IC- 
UC). For all non-zero and finite values of cooperation noise variances, our outer bound 
strictly improves upon the cut-set outer bound. We should remark here that the approach 
of dependence balance was also used in [12] to obtain an improved sum-rate upper bound 
for the Gaussian IC with common, noisy feedback from the receivers. 

Evaluation of our outer bounds for MAC-NF, MAC-UC and IC-UC is not straightforward 
since our outer bounds are expressed in terms of a union of probability densities of three 
random variables, one of which is an auxiliary random variable. Moreover, these unions 
are over all such densities which satisfy a non-trivial dependence balance constraint. We 
overcome this difficulty by proving separately for all three models in consideration, that it is 
sufficient to consider jointly Gaussian input distributions, satisfying the dependence balance 
constraint, when evaluating our outer bounds. The proof methodology for showing this claim 
is entirely different for each of the cases of noisy feedback and user cooperation models. 
In particular, for the case of MAC-NF, we make use of a recently discovered multivariate 
generalization [13] of Costa's entropy power inequality (EPI) [14] along with some properties 
of 3 X 3 covariance matrices to obtain this result. On the other hand, for the case of MAC- 
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UC and IC-UC, we do not need EPI to show this result and our proof closely follows the 
proof of a recent result by Bross, Lapidoth and Wigger [15], [16] for the Gaussian MAC with 
conferencing encoders. The structure of dependence balance constraints for the channel 
models in consideration are of different form, which explains the different methodology of 
proofs. 

For the most general setting of MAC-GF and IG-GF, our outer bounds are expressed in 
terms of two auxiliary random variables. For the three channel models in consideration, i.e., 
MAC-NF, MAC-UC and IC-UC, we suitably modify our outer bounds to express them in 
terms of only one auxiliary random variable. These modifications arc particularly helpful in 
their explicit evaluation. We also believe that the proof methodology developed for evaluating 
our outer bounds could be helpful for other multi-user information theoretic problems. 

2 System Model 

2.1 MAC with Generalized Feedback 

A discrete memoryless two-user multiple access channel with generalized feedback (MAG- 
GF) (see Figure 1) is defined by: two input alphabets Xi and an output alphabet for 
the receiver y, feedback output alphabets and J^^j at transmitters 1 and 2, respectively, 
and a probability transition function p{y, yF^,yF^\xi, X2), defined for all triples (y, yp^, yp^) e 
y X ypi X yF2, for every pair (xi, X2) E Xi x X2. 

A [n, Ml, M2, Pe) code for the MAG-GF consists of two sets of encoding functions fu : 
Ml X y^pT^ Xi, f2i : M2 X y^pT^ X2 ioT i ^ 1, . . . ,n and a decoding function g : y^ ^ 
All X M.2- The two transmitters produce independent and uniformly distributed messages 
Wi e {1, . . . , Ml} and W2 G {1, . . . , M2}, respectively, and transmit them through n channel 
uses. The average error probability is defined as, Pg = Pr[(l^i, 1^2) 7^ (l^i,1^2)]- A rate 
pair (i?i, R2) is said to be achievable for MAC-GF if for any e > 0, there exists a pair of n 
encoding functions {/it}"=i, {/2i}r=i) ^nd a decoding function g : y^ — >■ Aii x Ai2 such that 
Ri < ^og{Mi)/n, i?2 < log(M2)/n and Pe < e for sufficiently large n. The capacity region of 
MAG-GF is the closure of the set of all achievable rate pairs (i?i, R2). 

2.2 IC with Generalized Feedback 

A discrete memoryless two-user interference channel with generalized feedback (IG-GF) (see 
Figure 2) is defined by: two input alphabets Xi and X2, two output alphabets iVi and y2 at 
receivers 1 and 2, respectively, two feedback output alphabets yp^ and yp^ at transmitters 1 
and 2, respectively, and a probability transition function p{yi, y2, yFnllFil^i, ^2), defined for 
all quadruples (yi, ^2, Z/Fi, Z/Fa) e x 3^2 x yp^ x yp^, for every pair {xi,X2) e Xi x X2. 

A (n, Ml, M2, Pe^\ Pe"^^) code for IG-GF consists of two sets of encoding functions fu : 
Ml X y^pT^ Xi, f2i : M2 X y^pT^ X2 ioT i — 1, . . . ,n and two decoding functions 
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Figure 1: The multiple access channel with generalized feedback (MAC-GF). 



gi ■ yi ^ M.1 and 92 '■ y2 ^ •M.2- The two transmitters produce independent and uniformly 
distributed messages Wi G {1. . . . , Mi} and W2 G {1, . . . , M2}, respectively, and transmit 
them through n channel uses. The average error probability at receivers 1 and 2 are defined 
as, Pi^'* = Pr[M^fe 7^ Wk] for k = 1,2. A rate pair {Ri, R2) is said to be achievable for IC-GF 
if for any pair ei > 0, 62 > 0, there exists a pair of n encoding functions {/2i}iLi) 
and a pair of decoding functions ((71, (72) such that Ri < log(Mi)/n, R2 < log(M2)/n and 
Pe'^^ ^ Cfc ioT sufficiently large n, for k = 1, 2. The capacity region of IC-GF is the closure of 
the set of all achievable rate pairs i?2)- 



3 Cut-set Outer Bounds 

A general outer bound on the capacity region of a multi-terminal network is the cut-set outer 
bound [17]. The cut-set outer bound for MAC-GF is given by 

^^MAC ^ : Ri < I{X^■Y,YF,\X2) (1) 

R2<I{X2;Y,Yf,\X,) (2) 
Ri + R2<I{Xi,X2;Y)} (3) 

where the random variables Xi, X2 and {Y, Yp^, Yp^) have the joint distribution 

p(xi,x2,y,yFi,yF2) ^ p{xi,x2)piy,yFi,yF2\xi,x2). (4) 
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Figure 2: The interference channel with generahzed feedback (IC-GF). 
The cut-set outer bound for IC-GF is given by 

CS'^ = {{Ri,R2) : Ri < I{X,,X2;Y,) (5) 

i?2</(Xi,X2;F2) (6) 

R,<I{X,;Y,,Y2,Yf,\X2) (7) 

R2 < I{X2;Y,,Y2,Yf,\X^) (8) 

Ri + R2<I{X^,X2;Y^,Y2)} (9) 

where the random variables Xi, X2 and (Fi, Y2, Yfi, Yp^) have the joint distribution 

p{xuX2,yuy2,yFi,yF2) = p{xuX2)p{yi,y2,yFi,yF2\xi,x2). (10) 

The cut-set bound is seemingly loose since it allows arbitrary correlation among channel in- 
puts by permitting arbitrary input distributions p{xi, X2). Using the approach of dependence 
balance, we will obtain outer bounds for MAC-GF and IC-GF which restrict the correspond- 
ing set of input distributions for both channel models. In particular, our outer bounds only 
permit those input distributions which satisfy the respective non-trivial dependence balance 
constraints. 

4 A New Outer Bound for MAC-GF 

Theorem 1 The capacity region of MAC-GF is contained in the region 



VB 



MAC 



{{Ri,R2) : Ri < I{X^;Y,Yf,\X2,T2) 
R2 < I{X2;Y,Yf,\X,,T,) 
Ri + R2< I{X,, X2; Y, Yf„Yf, |Ti, T2) 
Ri + R2<I(X,,X2;Y)} 



(11) 
(12) 

(13) 

(14) 
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where the random variables {Ti,T2, Xi, X2,Y,Yfi,Yf2) have the joint distribution 

p{ti,t2,Xi,X2,y,yFi:yF2) = P{tut2,Xi,X2)p{y,yFi:yF2\XuX2) (15) 

and also satisfy the following dependence balance bound 

IiX,;X2\T,,T2) < I{X^;X2\Yf„Yf„T,,T2) (16) 
The proof of Theorem 1 is given in the Appendix. 

5 A New Outer Bound for IC-GF 

Theorem 2 The capacity region of IC-GF is contained in the region 



VB'''^{{Ri,R2): 



Ri 


</(Xi,X2;yi) 


(17) 


R2 


<I{X^,X2;Y2) 


(18) 


Ri 


<I{X,;Y,,Y2,Yf,\X2,T2) 


(19) 


R2 


<IiX2■,Y^,Y2,YF,\X,,T,) 


(20) 


Ri 


+ R2< /(Xi, X2; Fi, Y2, Yf„ Yf, |Ti, T2) 


(21) 


Ri 


+ R2<I{X,,X2;Y,,Y2)} 


(22) 



where the random variables (Ti, T2, Xi, X2, li, ^2, Xfi, XF2) have the joint distribution 

p{ti, t2, xi,x2, yi, y2, yPi.yF^) = p(^i, ^2, xi, X2)p{yi, ^2, yFi,yF2\xi,x2) (23) 

and also satisfy the following dependence balance bound 

I{X,;X2\Ti,T2) < I{Xi;X2\Yf„Yf„T,,T2) (24) 

The proof of Theorem 2 is given in the Appendix. 

We note here that one can obtain fixed and adaptive parallel channel extensions of the 
dependence balance based bounds in a similar fashion as in [1]. The parallel channel ex- 
tensions could potentially improve upon the outer bounds derived in this paper. For the 
scope of this paper, we will only use Theorems 1 and 2. In the next three sections, we will 
consider specific channel models of MAC with noisy feedback, MAC with user cooperation, 
and IC with user cooperation and specialize Theorems 1 and 2 for these channel models. 
In particular, we will show that for these three channel models, it is sufficient to employ a 
single auxiliary random variable T, as opposed to two auxiliary random variables Ti and T2 
appearing in Theorems 1 and 2. 
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We should also remark here that dependence balance approach was first applied by Gast- 
par and Kramer for the Gaussian MAC with noisy feedback in [11] and for the Gaussian 
IC with noisy feedback (IC-NF) in [12]. An interesting Lagrangian based approach was 
proposed in [12] to partially evaluate the dependence balance based outer bound for the 
Gaussian IC-NF and it was shown that dependence balance based bounds strictly improve 
upon the cut-set outer bound. For this reason, we do not consider the Gaussian IC-NF in 
this paper. 

6 Gaussian MAC with Noisy Feedback 

We first consider the Gaussian MAC with noisy feedback (see Figure 3). The channel model 
is given as, 

Y = Xi+X2 + Z (25) 
Yf^=Y + Zi (26) 
Yf, = Y + Z2 (27) 

where Z,Zi and Z2 are independent, zero-mean, Gaussian random variables with variances 
cr|, (t|^ and cxl^, respectively. Moreover, the channel inputs are subject to average power 
constraints, -E[Xf] < Pi and -E[Ar|] < P2. Note that the channel model described above has 
a special probability structure, namely, 

p(y,yFi,l/F2ki,a;2) = p{y\xi,X2)p{yFt\y)p{yF2\y) (28) 

For any MAC-GF with a transition probability in the form of (128|) . we have the following 
strengthened version of Theorem 1. 

Theorem 3 The capacity region of any MAC-GF, with a transition probability in the form 
of ( fl^) . is contained in the region 

VB^'^f = {{RuR2) : Ri < I{X,;Y\X2,T) (29) 

R2<I{X2;Y\X^,T) (30) 
Ri + R2<I{X,,X2;Y\T)} (31) 

where the random variables (T, Xi, X2, F, Yp^, YpJ have the joint distribution 

p{t,xi,x2,y,yFi,yF2) = p{t,xi,x2)p{y\xi,x2)p{yFAy)p{yF2\y) (32) 
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Figure 3: The Gaussian MAC with noisy feedback. 



and also satisfy the following dependence balance hound 



■W2 



/(Xi;X2|T) </(Xi;X2|y^,,y^„T) 



(33) 



where the random variable T is subject to a cardinality constraint \T\ < \Xi\\X2\ + 3. 

The proof of Theorem 3 is given in the Appendix. 

In Section 10, we will show that it suffices to consider jointly Gaussian (T, Xi,X2) satis- 
fying ( l33l) when evaluating Theorem 3 for the Gaussian MAC with noisy feedback described 
in 



7 Gaussian MAC with User Cooperation 

In this section, we consider the Gaussian MAC with user cooperation [2] , where each trans- 
mitter receives a noisy version of the other transmitter's channel input. The user cooperation 
model (see Figure 4) is a special instance of a MAC-GF, where the channel outputs are de- 
scribed as, 

Y = v^Xi + ^/hioX2 + Z (34) 
Yf, = v^X2 + Zi (35) 
Yf2 = v^Xi + Z2 (36) 

where Z,Zi and Z2 are independent, zero-mean, Gaussian random variables with variances 
cr|, 0"!^ and o"!^, respectively. The channel gains /iio, ^20? ^12 and /i2i are assumed to be 
fixed and known at all terminals. Moreover, the channel inputs are subject to average power 
constraints, -EfXf] < Pi and -E[X|] < P2. Note that the channel model described above has 
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Figure 4: The Gaussian MAC with user cooperation. 



a special probability structure, namely, 



-Wi 
■W2 



p{y,yFi,yF2\xi,x2) = p{y\xi,x2)p{yFAx2)p{yF2\xi 



(37) 



For any MAC-GF with a transition probabihty in the form of fl37j) . we have the following 
strengthened version of Theorem 1. 

Theorem 4 The capacity region of any MAC-GF with a transition probability in the form 
of [3^, is contained in the region 



VB^ji'' = {iRi,R2) : Ri < I{X,;Y,Yf,\X2,T) 

R2<I{X2;Y,Yf,\X,,T) 
Ri + R2 < I{X,,X2;Y,Yf„Yf,\T) 
Ri + R2<I{X,,X2;Y)} 

where the random variables (T, Xi, X2, F, Yfi, ^^2) have the joint distribution 

pit, xi, X2, y, yFi,yF2) = pit, xi,x2)piy\xi, x2)piyF^ \x2)piyF2\xi) 

and also satisfy the following dependence balance bound 

IiXr,X2\T)<IiXi-X2\YF,,YF,,T) 

where the random variable T is subject to a cardinality constraint \T\ < \Xi\\X2\ + 3. 



(38) 
(39) 
(40) 
(41) 



(42) 



(43) 



The proof of Theorem 4 is given in the Appendix. 

In Section 11, we will show that it suffices to consider jointly Gaussian (T, Xi, X2) satisfy- 
ing ( l43ll when evaluating Theorem 4 for the Gaussian MAC with user cooperation described 
in (I3l-(I36I). 
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8 Gaussian IC with User Cooperation 



In this section, we will evaluate our outer bound for a user cooperation setting [6], [7], where 
the transmitters receive noisy versions of the other transmitter's channel input. The user 
cooperation model (see Figure 5) is a special instance of an IC-GF, where the channel outputs 
are described as, 



Yi = Xi + VbX2 + Ni (44) 

Y2 = v^Xi + X2 + N2 (45) 

Yf, = Vh'iX2 + Z, (46) 

Yf, = v^Xi + Z2 (47) 



where Ni, N2, Zi and Z2 are independent, zero-mean, Gaussian random variables with vari- 
ances a%_^,a%^, (t|^ and crf^, respectively. The channel gains a,b,hi2 and /i2i are assumed 
to be fixed and known at all terminals. Moreover, the channel inputs are subject to average 
power constraints, -E[X^] < -Pi and -E[X|] < P2. Note that the channel model described 
above has a special probability structure, namely, 

Piyi,y2,yF^,yF2\XuX2) = p{yuy2\Xu X2)p{yFi\x2)p{yF2\Xl) (48) 

For any IC-GF with a transition probability in the form of fj48l) . we have the following 
strengthened version of Theorem 2. 

Theorem 5 The capacity region of any IC-GF with a transition probability in the form of 
( f^gp , is contained in the region 

VBjfc = {{Ri,R2) : Ri < /(Xi,X2;Fi) (49) 

R2<I{XuX2;Y2) (50) 

Ri < I{X^;Y^,Y2,Yf,\X2,T) (51) 

R2 < I{X2;Y^,Y2,YfAX,,T) (52) 

Ri + R2< I{Xi,X2; Fi, Y2, Yf„Yf, \T) (53) 

R, + R2<I{XuX2;Y,,Y2)} (54) 

where the random variables {T, Xi, X2,Yi,Y2,Yf^,Yf2) have the joint distribution 

p{t, Xi, X2, Vu 2/2, yF^^VF^) = P{t, Xi,X2)p{yu 2/2^1, X2)p{yFi \x2)p{yF2\Xl) (55) 

and also satisfy the following dependence balance bound 

I{X,;X2\T) < I{X^,X2\Yf,,Yf„T) (56) 



11 





Encoder 1 


) 


y- 


Decoder 1 


^ 




Z2 \x^/ 










Encoder 2 




Decoder 2 




X2 





N2 

Figure 5: The Gaussian IC with user cooperation. 

where the random variable T is subject to a cardinality constraint \T\ < \Xi\\X2\ +3. 

The proof of Theorem 5 is given in the Appendix. 

In Section 12, we will show that it suffices to consider jointly Gaussian (T, Xi,X2) satis- 
fying (!56|) when evaluating Theorem 5 for the Gaussian IC with user cooperation described 

in dSD-dlZD. 



9 Outline for Evaluating VB^f, VB^^i^ and VB\^c 



MAC 



In this section, we outline the common approach for evaluation of our outer bounds, "DBj^p 



for the Gaussian MAC with noisy feedback, VBff^'" for the Gaussian MAC with user- 



cooperation and VB^^fj for the Gaussian IC with user-cooperation. The main difficulty in 
evaluating these bounds is to identify the optimal selection of joint densities of {T, Xi, X2). 
Our aim will be to prove that it is sufficient to consider jointly Gaussian (T, Xi, X2) satisfying 
( 133|) for MAC with noisy feedback, f H3l) for MAC with user cooperation, and (156|) for IC with 
user cooperation, respectively, when evaluating the corresponding outer bounds. 

First note that the three outer bounds, namely VB^p'^\ VBff^'" and T>B\^(j have a similar 
structure, i.e., all outer bounds involve taking a union over joint densities of (T, Xi,X2) 
satisfying the constraints (l33l) . (H3l) and (1561) . respectively. Let us symbolically denote these 
constraints as a variable (DB), where (DB) = fl33|) for MAC with noisy feedback, {DB) = 
fH3|) for MAC with user cooperation, and {DB) = fl56|) for IC with user cooperation. 

We begin by considering the set of all distributions of three random variables (T, Xi, X2) 
which satisfy the power constraints, i?[X^] < Pi and i?[X|] < P2. Let us formally define 
this set of input distributions as 

V = {p{t,x,,X2) : ^[X^] < P,,E[X^] < P2} 

For simplicity, we abbreviate jointly Gaussian distributions as J^Q and distributions which 
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are not jointly Gaussian as HG- We first partition V into two disjoint subsets, 



Vg = {p{t,x^,X2) e V : (T,Xi,X2) arc jg] 
Vng = {p{t, xi,X2)eV: {T, Xi, X2) are AfG} 

We further individually partition the sets Vg and Vng, respectively, as 

Vg^ = {p{t,xi,X2) e Vg : (T,Xi,X2) satisfy (DB)} 

-pDB = {p(^t,Xi,X2) e Vg ■■ {T,Xi,X2) do not satisfy (DB)} 

and 

T^nE = {pit,xi,X2) e Vng ■ iT,Xi,X2) satisfy (DB)} 

'PM = {P{i. ^1, ^2) e Vng ■ {T, X^, X2) do not satisfy (DB)} 

Finally, wc partition the set V^g ^^^^ ^'^^ disjoint sets V^g^"'^ T^ng^'^^ '^i^^ T^ng — 

I^NG \J I^NG ' "^^ 

, Xi, X2) e V^G '■ covariance matrix of p{t, Xi, X2) is Q and there 
exists a J^G {Tg- Xig,-^2g) with covariance matrix Q satisfying (DS)} 
V^G^^^ ~ {p(^i ^2) G T^NG '■ covariance matrix of p{t, Xi,X2) is Q and there 

does not exist a JG {Tg,Xig,X2g) with covariance matrix Q satisfying (-D-B)} 

So far, wc have partitioned the set of input distributions into five disjoint sets: Vq^, 
Vg^ , V^Q^'^\ V^Q^^^ and V^g- '^'^ visualize this partition of the set of input distributions, 
see Figure 6. It is clear that the input distributions which fall into the sets Vq^ and V^q 
need not be considered since they do not satisfy the constraint {DB) and do not have 
any consequence when evaluating our outer bounds. Therefore, we only need to restrict our 
attention on the three remaining sets Vq^ , V^g^"'\ and V^q^^^ ^-^-i those input distributions 
which satisfy the dependence balance bound. 

We explicitly evaluate our outer bound in the following three steps: 

1. We first explicitly characterize the region of rate pairs provided by our outer bound 
for the probabihty distributions in the set Vq^ . 

2. In the second step, wc will show that for any input distribution belonging to the set 
V^Q^°'\ there exists an input distribution in the set Vq^ which yields a set of larger 
rate pairs. This leads to the conclusion that we do not need to consider the input 
distributions in the set V^§''"'^ in evaluating our outer bound. 

3. We next focus on the set V^q^^^ show that for any non-Gaussian input distribution 
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Figure 6: A partition of the set of input distributions V. 



p{t, Xi,X2) e 'Png ' construct a jointly Gaussian input distribution satisfying 

(DB), i.e., we can find a corresponding input distribution in Vq^, which yields a set 
of rates which includes the set of rates of the fixed non-Gaussian input distribution 

p{t, Xi,X2). 

The main step in evaluating our outer bounds is step 3 described above. The proofs 
of step 3 for noisy feedback and user cooperation models are entirely different and do not 
follow from each other. The evaluation in step 1 is slightly different for all three settings, 
also owing to the channel models. Hence, we will separately focus on these models in the 
following three sections. Contrary to steps 1 and 3, step 2 is common for all channel models. 
Therefore, we first present the common result for all channel models here. In step 2, we 
consider any non-Gaussian input distribution p(t, Xi, X2) in V^q^"'^ with a covariance matrix 
Q. For such an input distribution, we know by the maximum entropy theorem [17], that the 
rates provided by a jointly Gaussian triple with the same covariance matrix Q are always at 
least as large as the rates provided by the chosen non-Gaussian distribution. Therefore, for 
any input distribution in V^q^"''' , there always exists an input distribution in Vq^, satisfying 
{DB), which yields larger rates. This means that we can ignore the set T^ivG^""* altogether 
while evaluating our outer bounds. 

To set the stage for our evaluations in steps 1 and 3 for the three channel models, let us 
define Q as the set of all valid 3x3 covariance matrices of three random variables (T, Xi, X2). 
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A typical element Q in the set Q takes the following form, 



Q = E[{X,X2T){X,X^Tf] 

( Pi P12VIW2 PitVPJ^ \ 

\ PiTyPiPr P2TVP2PT Pt / 



(57) 



A necessary condition for Q to be a valid covariance matrix is that it is positive semi- definite, 
i.e., det{Q) > 0. This is equivalent to saying that. 



det(Q) = PiPsPtA > 



(58) 



where we have defined for simplicity. 



A = 1 - - Pit - pIt + '^PitP2tPi2 



(59) 



10 Evaluation of VB^^^ 

In this section we explicitly evaluate Theorem 3 for the Gaussian MAC with noisy feedback 
described by (!25!)- (l271) in Section 6. We start with step 1. We consider an input distribution 
in Vq^, i.e., a jointly Gaussian triple {Tq, Xiq, X2g) with a covariance matrix Q. Let us 
first characterize the set of rate constraints for this triple. It is straightforward to evaluate 
the three rate constraints appearing in fl29|) - fl3T|) for this input distribution 

Pi < /(Xig; Y\X2g, Tg) = hog (1 + (60) 



2 V (^i 

1, A , MQ) 



R2 < I{X2g; Y\X,g, Tg) = -log ( 1 + ] (61) 



'z 



Pi + P2 < nX,G, X2g; Y\Tg) = hog (1 + ] (62) 



'z 



where we have defined 



AP 

/i(Q) = Var(XiG|X2G, Tg) = (^3^ (63) 

APo 

/2(Q) = Var(X2G|XiG,TG) = (^3^ (64) 
MQ) = Var(XiGlTG) + Var(X2G|TG) + 2Coy{X^g, X2g\Tg) 



[1 - pIt)P^ + (1 - pI^)P2 + 2(pi2 - PitP2t)Vi\P2 (65) 
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Finally, evaluating the constraint in (133|) . we conclude that this input distribution satisfies 
the constraint in (133!) iff, 



/3(g) < /i(Q) + /2(g) + , ^^^^^f^^g^ ^ (66) 



To summarize, the set of rate pairs provided by an input distribution in Vq^ , with a covari- 
ance matrix Q, are given by those in (!60|) - (!62|) . where fi{Q), i = 1,2, 3, in those inequalities 
are subject to the constraint in fl66l) . As we have discussed earlier, from evaluation of step 
2 in Section 9, we know that all rate pairs contributed by input distributions in V^q^'^^ are 
covered by those given in Vq^ . 

We now arrive at step 3 of our evaluation. Consider any input distribution p{t, xi,X2) 
in V^Q^''^ with a covariance matrix Q. By the definition of the set V^q^'^\ we know that Q 
does not satisfy (l33i) . which implies 

HQ) > HQ) + /2(Q) + A(^)yg) (67) 

We also note that for any (T, Xi,X2) with a covariance matrix Q, 

1, (. , /i(g) 



< /(Xi;F|X2,T) < -log ( 1 + ) (6^ 



'z 



R2 < Y\X,, T) < hog (l + ) (69) 



Ri + R2< /(Xi, X2; Y\T) < hog + ^^^^ (70) 

which is a simple consequence of the maximum entropy theorem [17]. Note that so far, we 
have not used the fact that the given non-Gaussian input distribution satisfies the dependence 
balance constraint in (15^ . We will now make use of this fact by rewriting as follows, 

< I{Xv,X2\Yf^,Yf^,T) - /(Xi;X2|r) (71) 
= /(Xi; Yp^,Yp^\X2, T) - /(Xi; F^,, F^,|T) (72) 
= h{YF„YF,\X^, T) + h{YF„YF,\X2, T) - /i(y^„ y^,|T) - h{YF„YF,\X^, X^, T) (73) 

We express the above constraint as, 

h{YF, , Yf, \T) + h{YF, , Yf, |Xi, X2, T) < h{YF, , Yf, |Xi, T) + h{YF, , Yf, IX2, T) (74) 

Before proceeding, we state a recently discovered multivariate generalization [13] of Costa's 
EPI [14]. 
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Lemma 1 For any arbitrary random vector Y G M^, independent of\& M^, where \ is a 
zero-mean, Gaussian random vector with each component having unit variance, the entropy 
power Ar(Ai/2Y + V) is concave in A, where the entropy power is defined as 

and A is a diagonal matrix with components (Ai, A2). 

We can therefore write for any pair of diagonal matrices Ai, A2 and for any fj, G [0, 1], 
I^N{Ay^Y + V) + (1 - /i)iV(A^/'Y + V) < NiifiA, + (1 - /i)A2)'/'Y + V) (76) 
We start by obtaining a lower bound for the first term h(Yp-^,Yp^\T) in (17II) . 

h{YF„YF,\T) = J fT{t)h{Y + Z,,Y + Z2\T = t)dt (77) 

> / /T(t)^log ((2vre) V|^4. + 27re(4, + 4je^'^(^l^=*)) dt (78) 

> hog {{2neralal + 2ne{al + afje^'^^^l^)) (79) 

where 0781) follows from the conditional version of Lemma 1, by selecting the following Ai, 
A2 and /i 

f 1^ ^\ . f ^ 



where 



n = ^^^-2^ (81) 

and 

and by making the following substitutions, 

V^ = ^, V, = ^ (83) 

where V = [Vi V'l^ and Y = YY ■ A derivation of fl78l) is given in the Appendix. Next, 
( 1791) follows from the fact that log(e^ci + C2) is convex in x for ci, C2 > and a subsequent 
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application of Jensen's inequality [17^. 

We next obtain an upper bound for the right hand side of (17^ by using the maximum 
entropy theorem as, 

h{YF,,YF,\Xu T) + h{YF,,YF,\X2, T) 

< \\og ((2vre)^(A(Q)(4, + 4j + ^)(/2(Q)(4. + 4j + v)) (84) 
where we have defined 

^ = 4i4. + + (85) 

Now, using (17il) . (1791) and (l84l) . we obtain an upper bound on h{Y\T) as follows, 

Mr|T)<llog((27re)(4 + /(g))) (86) 
where we have defined for simplicity, 

f{Q) = MQ) + h{Q) + , ^'^^^l^^^J (87) 

Using (!86|) . we obtain an upper bound on the sum-rate J(Xi, X2; V|T) for any non-Gaussian 
distribution in V^q^^^ as, 

Ri + R2< /(Xi, X2; r |T) < iog (^1 + (88) 

Comparing with flTOj) and using the fact that Q satisfies (l67j) . i.e., f{Q) < fsiQ), we have 
the following set of inequalities. 



R^ + R,< /(Xi, X,; Y\T) < hog (^1 + < hog (^1 + ^) 



(89) 



This leads to the observation that a combined application of the EPI and the dependence 
balance bound yields a strictly smaller upper bound for I{Xi, X2; Y\T) for any distribution 
in V^Q^^^ than the one provided by the maximum entropy theorem. Therefore, the rate 
pairs contributed by an input distribution in V^q^^^ with a covariance matrix Q are always 
included in the set of rate pairs expressed by (!68l) . (!69|) and (1881) . where f{Q) is defined in 



^ We should remark here, that an appHcation of the regular form of vector EPI yields the following trivial 
lower bound on /i(ypj^ , Yp^ |T) and therefore, the new EPI is crucial for this step. 

h{Yp„YpjT) > ilog((27re)2a|^4J 
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We now arrive at the final step of our evaluation where we will show that for this input 
distribution in 'P^q^^\ we can always find an input distribution in Vq^ , with a set of rate 
pairs which include the set of rate pairs expressed by (1681) . ( |69l) and (1881) . In particular, we 
will show the existence of a valid covariance matrix S for which the following inequalities 
hold true, 

MQ) < fi{S) (90) 

/2(g) < f2iS) (91) 
fiQ) < US) (92) 

and 

MS) = MS) + MS) + / f ^^Iffl X (93) 

+ (4?+5,)j 

Inequalities in fl90|) - fl92l) will guarantee that a Gaussian input distribution with covariance 
matrix S yields a larger set of rate pairs than the set of rate pairs expressed by ( l68l) , f l69l) and 
( IHHl) and the equality in ( l93l) guarantees that this input distribution satisfies the dependence 
balance constraint with equality, hence it is a member of the set Vq^. 

Before showing the existence of such an S, we first characterize the set of covariance 
matrices Q which satisfy (l^7|) . First recall that for any Q to be a valid covariance matrix, 
we had the condition det(Q) > which is equivalent to A > 0, which amounts to 

1 - P12 - Pit - pIt + 2pitP2tPi2 > (94) 

In particular, it is easy to verify that for any given fixed pair (pir,P2r) G [—1, 1] x [—1, 1], 
the set of pi2 which yield a valid Q are such that, 

P1TP2T - A < P12 < P1TP2T + A (95) 

where we have defined 



A = y/(l-p?^)(l-p2^) (96) 

We now consider two cases which can arise for a given covariance matrix Q. 

Case 1. Q is such that pi2 = PitP2T — for some a G [0, A]: This case is rather trivial 
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and the following simple choice of 5* works, 

p[t = Pit, pIt = P2T (97) 

Pl2 = PlTp2T (98) 

Clearly, this S satisfies the dependence balance bound. Moreover, the following inequalities 
hold as well, 

/i(g)</i(5) = (l-p?^)Pi (99) 

/2(g)</2(5) = (l-p^T)A (100) 

f{Q) < HQ) (101) 



= (1 - pfy)Pi + (1 - p'^t)P2 - 2a VAP2 (102) 

< {1-pIt)Pi + {1~pIt)P2 (103) 

= US) (104) 

Case 2. Q is such that pi2 = PitP2T + «0; ^oi some ao G (0, A] and Q satisfies (167|) : For 
this case, we will construct a valid covariance matrix S as follows, 

p[t = Pit, pir = P2T (105) 

p[2^ = PitP2T + Ci* , for some < a* < ao (106) 

We define a parameterized covariance matrix Q{a) with entries, 

PiT(tt) = Pit, P2T(tt) = P2r (107) 

Pi2(a) = PirP2r + a (108) 

where < a < ao- We now define a function of the parameter a of a valid covariance matrix 
Q{a) as, 

9{a) = h{Q{a)) + h{Q{a)) + /^(^("))J^(g(^)) _ f^^Q^^)) (109) 
Now note the fact that 

.(0) ^ ^^-Ar)i^;/M , „ (,,0) 



We are also given that Q satisfies fl67p for some ao, which implies that, 

^?(ao) < (111) 
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Now, we take the first derivative of the function g{a), to obtain, 



da V (1 - pIt) ' (1 - pIt) J (n^ + "S^h 



< 



which imphes that g{a) is monotonically decreasing in a. This implies that there exists an 
a* G (0, ao) such that g{a*) = C§|. We use this a* to construct our new covariance matrix S 



as follows, 



Pit = Pit, p2T = P2T (112) 

Plf = PITP2T + ex* (113) 



It now remains to check wether 5* satisfies the four conditions in ( ]90l) - (!93l) . The condition (l93l) 
is met with equality, since we have g{a*) = 0. Moreover, fi{Q) = fi{Q{ao)) < fi{Q{a*)) = 
fi{S) since fi{Q{a)) is monotonically decreasing in a for a G [0, A]. Similarly, we also have 
/2(g) < f2{S). Finally, 

f{Q) = HQ) + /2(g) + /^(^)/^(g) (114) 
< /i(5) + /2(5) + f . (115) 

= /3(5) (116) 

This shows the existence of a valid covariance matrix S* which satisfies (1331) and yields a 
set of rates which includes the set of rates of the given non- Gaussian distribution with the 
covariance matrix Q. 

Above two cases show that for any non-Gaussian distribution p(t, Xi,X2) in the set V^q'^^\ 
we can always find a jointly Gaussian triple (T^, Xig, X2g) in Vq^ that yields a set of rates 
subsuming the set of rates of the given non-Gaussian distribution. This consequently com- 
pletes the proof of the statement that it is sufficient to consider jointly Gaussian (T, Xi, X2) 
in Vq^ when evaluating our outer bound. 

The dependence balance based outer bound can now be written in an explicit form as 



^We should remark here that the existence of an a* £ {0,ao), with g{a*) — can also be proved 
alternatively by invoking the mean value theorem, since we have g{0) > 0, g{ao) < and g{a) is a continuous 
function of a. Monotonicity of g{a) in fact proves a stronger statement that such an a* exists and is also 
unique. 
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follows, 

fi{Q) 



VB 



R2 < -log I 1 H 2~~ 



i?l + i?2<il0gfl + ^ 

4 



where Q^"^ is the set of 3 x 3 covariance matrices of the form (!57|) satisfying, 



(T-7 0"-7 



where 



APi 



and 



;ii7) 



/3(g) < HQ) + /2(g) + , f'^^^^/^J ^ (118) 



/i(g) = 7^ — (119) 



/3(g) = (1 - pIt)Pi + (1 - P2r)^2 + 2(pi2 - PitP2t)^/P^2 (121) 



A = 1 - - P2T - P12 + 2piTP2TPl2 (122) 



where pi2,PiT and p2T are all in [—1, 1]. 

The cut-set outer bound given in is evaluated for the Gaussian MAC with noisy 

feedback described in f l25|) - fl27|) as 



P qMAC 



NF — 

pe[o 



;fo,il I \ z / 



1, ^f. Pi + P2 + 2pVPj^ 

4 



Ri + R2< 2log ( 1 + \2 ] } (123) 



We briefly mention what our outer bound gives for the the two limiting values of the 
:kward noise variances cr|^ and crf^. 

1. (t|^, crf^ — > : this case corresponds to the Gaussian MAC with noiseless feedback and 
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the constraint flllSp simplifies to 



fsiQ) < h{Q) + /2(g) + M^l^ (124) 



which is simply stating that the sum-rate constraint should be at most as large as the 
sum of the individual rate constraints, i.e., another equivalent way of writing is 



-lo^[l + ^) < Jlog (1 + ^ j + jlog ^1 + ^ 1 (125) 



This is the same constraint as obtained by Ozarow in [10], and our outer bound coin- 
cides with the cut-set bound and yields the capacity region of the Gaussian MAC with 
noiseless feedback. 

2. (y\^,a\^ 00: this case corresponds to very noisy feedback and our outer bound 
should collapse to the no-feedback capacity region of the Gaussian MAC. For this case, 
the constraint flllSp simplifies to, 

/3(Q)</i(g) + /2(g) (126) 

On substituting the values of fi{Q), f2{Q) and fsi^Q) in the above inequality, we obtain 

< (128) 

where the last inequality comes from the fact that for any valid covariance matrix, 
A < A^. This implies that the dependence balance bound only allows such covariance 
matrices Q for which pi2 < PitP2T- But we know already from fp7l) - flU51) that we can 

(S) 

always find an S for which we can select — P1TP2T, which satisfies the dependence 
balance bound and yields larger rates than any Q with pi2 < PitP2T- Thus, we only 
need to restrict our attention to those matrices Q for which pi2 = PitP2T- Such 
covariance matrices Q correspond to those jointly Gaussian triples which satisfy the 
Markov chain Xi —>■ T X2. This can be observed by noting that for any jointly 
Gaussian [T, Xi, X2), with a covariance matrix Q, the condition /(Xi;X2|T) = 
holds iff Var(Xi|T) = Var(Xi|X2, T), which is equivalent to pu = PitP2T- Proof of 
this statement is immediate by noting that for a jointly Gaussian triple, we have 

Therefore, T can be interpreted simply as a timesharing random variable and our outer 
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bound yields the capacity region of the Gaussian MAC without feedback. 

Figure 7 illustrates VB^p'" , the cut-set bound and the capacity region without feedback 
for the cases when (t|^ = a^^ = 2, 5 and 10, where Pi = P2 = = 1- Figure 8 illustrates 
VB^p'" , the cut-set bound, the capacity region without feedback and an achievable rate 
region based on superposition coding [5] for the case when cr|^ = a^^ = 0.3 and Pi = P2 = 
a| = l. 

10.1 Remark 

For the special case of Gaussian MAC with common, noisy feedback, where 

Yf, = Yf, = Y + V (130) 

the evaluation of T>B^p'" follows in a similar manner as in the case of different noisy feedback 
signals. The only difference arises in the application of the EPI. In particular, the regular EPI 
[17] suffices to provide a non-trivial upper bound on /(Xi,X2; y\T) than the one provided 
by the maximum entropy theorem [17]. The remainder of the proof of evaluation of our 
outer bound for this channel model follows along the same lines as the proof for different 
noisy feedback signals. The final expressions of outer bounds for these two channel models 
only differ over the constraint (11181) . For the case of common, noisy feedback, the set Q^^ 
comprises of 3 x 3 covariance matrices of the form (1571) satisfying, 

HQ) < h{Q) + /2(Q) + ^'^fl'^?, (131) 

Now consider the Gaussian MAC with different noisy feedback signals Yp^ and Yp^ at the 
transmitters 1 and 2, respectively. If the variances of feedback noises Zi and Z2 are such 
that, 0"!^ = = CTy, then the dependence balance constraint (I118P simplifies as 

/3(g) < fm + /2(g) + 4^^!^ (132) 

This implies that if a covariance matrix Q satisfies the constraint (11311) . then it also satisfies 
(11321) but the converse statement may not always be true. This means that the resulting outer 
bound for the Gaussian MAC with common noisy feedback, with feedback noise variance 
ay can be strictly smaller than the resulting outer bound for Gaussian MAC with different 
noisy feedback signals, when the feedback noise variances are (t|^ = = ay. 
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Figure 7: Illustration of outer bounds for Pi = P2 = cr| 



1 and = (t|^ = 2, 5, 10. 
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Figure 8: Illustration of outer bound and an achievable region based on superposition coding 
for Pi = P2 = (x| = 1 and = (t|^ = 0.3. 
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11 Evaluation of V8f^^^ 

In this section we will explicitly evaluate Theorem 4 for the Gaussian MAC with user coop- 
eration described by fl3^ - fl5Bl) in Section 7. We start with step 1 and characterize the set of 
jointly Gaussian triples (To, Xig, X2g) in Vq^. For this purpose, we rewrite (H3!) as follows, 

< I{X,;X2\Yp„Yf„T) - /(Xi;X2|T) (133) 
= /(Xi; Yf„ Yf,\X2, T) - /(Xi; Yf,,Yf, \T) (134) 
= /i(F^„F^,|Xi,T) + h{YF^,YF,\X2,T) - h{YF^,YF,\T) - h{YF,,YF,\X^, X^^T) (135) 

and express the above constraint as follows, 

h{YF,,YF,\T) + h{YF,,YF,\X^, X2, T) < h{YF,, Yf,\X,, T) + h{YF,,YF,\X2, T) (136) 

Making use of the following equalities, 

h{YF,,YF,\X^,X2,T) = ^log((27re)V|^4J (137) 
hiYF„YF,\X,,T) = ilog((27re)4J +/i(y^jXi,T) (138) 
h{YF„YF,\X2,T) = hog {{27ie)crl) + h{YF,\X2,T) (139) 

we obtain a simplified expression for fll36p as, 

h{YF„YF,\T) < h{YF,\XuT) + h{YF,\X2,T) (140) 

We further simplify (I140p as follows, 

< h{YF,\Xi,T) + h{YF,\X2,T) - h{YF„YF,\T) (141) 

= h{YF,\X,,T) + h{YF,\X2,T) - h{YF,\T) - h{YF,\YF„T) (142) 

= -I{Yf,;X,\T) + h{YF,\X2,T) - h{YF,\YF„T) (143) 

= -I{Yf,;X,\T) + h{YF,\X2,YF„T) - h{YF,\YF,,T) (144) 

= -I{Yf,;Xi\T) - I{Yf,; X2\Yf,,T) (145) 

where fll44p follows from the Markov chain Yp^ — > X2 (T, Yf2)- Therefore, the dependence 
balance constraint in (H3l) is equivalent to following two equalities, 

/(F^,;Xi|T) = (146) 
I{Yf,;X2\Yf,,T) = (147) 
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Next, we show that if any jointly Gaussian triple {T, Xi, X2) satisfies the constraints (11460 - 
(11471) then it satisfies the Markov chain Xi — T ^ X2. Conversely, we will show that if any 
jointly Gaussian triple (T,Xi,X2) satisfies Xi ^ T ^ X2, then it satisfies (ll46p -( fT47j) . 

We start by evaluating (I146p and (11471) for a jointly Gaussian {Tc, Xic, ^20) which is 
equivalent to. 



= IWh2iX2G + Zi- XigITg) (148) 
= /( v^XiG + Z2; X2G\Vh^iX2G + Zi, Tg) (149) 

These equalities are equivalent to 

Cov(Xig,X2g|Tg) = (150) 
Using the same argument as in (I129p . we obtain the following condition 

Pl2 = plTp2T (151) 

This implies that a jointly Gaussian triple satisfies (I146p -( |T^7I) iff pi2 = PitP2T- 

On the other hand, consider any jointly Gaussian triple (T^, X2g), with a covari- 
ance matrix Q which satisfies the Markov chain Xig Tq ^ X2g- This is equivalent to 
I{Xig; X2g\Tg) = 0, which is equivalent to 

P12 = P1TP2T (152) 

This implies that if a jointly Gaussian triple (T, Xi,X2) satisfies the Markov chain Xi 
T —>■ X2, then it satisfies (11520 and therefore it also satisfies (ll46p -( fTTni and vice versa. As 
a consequence, we have explicitly characterized the set Vq^, i.e., it comprises of only such 
jointly Gaussian distributions, (T^, Xig, X2g), for which Xig — > ^g — > X2g- 

We can now write the set of rate pairs provided by our outer bound for a jointly Gaussian 
triple (Tg,Xig,X2g) in the set Pg^ as 

Ri < I{X,g;Y,Yp,\X2g,Tg) (153) 

R2<I{X2g;Y,Yf,\X^g,Tg) (154) 

Ri + R2< nXiG, X2g; Y, Yf,,Yf,\Tg) (155) 

Ri + R2<I{X,g,X2g;Y) (156) 

where (Tg,Xig,X2g) satisfies the Markov chain Xig —^Tg^ ^2G- Moreover, from the 
evaluation of step 2 in Section 9, we know that all rate pairs contributed by input distribu- 
tions in P^g''"^ covered by those given in Vq^. Therefore, we do not need to consider 
the set V^Q^"'^ in evaluating our outer bound. 
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We now arrive at step 3 of the evaluation of our outer bound where we will show that 
for any non-Gaussian input distribution p{t, xi,X2) G V^q'^^\ we can always find an input 
distribution in Vq^ , with a set of rate pairs which include the set of rate pairs of the 
fixed non-Gaussian input distribution p(t,a;i,X2). Consider any triple (T, Xi,X2) with a 
non-Gaussian input distribution p{t^Xi^X2) G "^NCi^^^ with a valid covariance matrix Q. 
By the definition of the set 'P^q^^\ and as a consequence of fllSip . this covariance matrix 
has the property that pi2 7^ PitP2T- Moreover, this non-Gaussian distribution satisfies the 
dependence balance bound, i.e., it satisfies (11461) and f ll47p . For our purpose, we only need 
mm . Since I{Yf,;Xi\T) = 0, this implies 



E[Wh2iX2 + Z,)X,\T] = E[,/hiX2 + Z^\T]E[X,\T] (157) 

= ^,E[X2\T]E[X^\T] (158) 



on the other hand, we also have -E[(V/i2i-^2 + Zi)Xi\T] = \/Ti2iE[XiX2\T], which implies 

E[XiX2\T] = E[X2\T]E[Xi\T] (159) 
We will now construct another triple (T', Xi, X2) with a covariance matrix 5* by selecting 

T' = E[Xi\T] (160) 



This particular selection is closely related to the recent work of Bross, Lapidoth and Wigger 
[15] where it was shown that jointly Gaussian distributions are sufficient to characterize 
the capacity region of Gaussian MAC with conferencing encoders. Although, we should 
also remark that when evaluating our outer bound for user cooperation, we do not have a 
conditionally independent structure among (T, Xi,X2) to start with. This structure arises 
from the dependence balance constraint fH51) . permiting us to use this approach. 

Returning to (11601) . we note that T' is a deterministic function of T and therefore, fol- 
lowing is a valid Markov chain. 

T' ^ T ^ (Xi,X2) ^ {Y,Yp„Yp,) (161) 

We will now obtain the off diagonal elements of the covariance matrix S of the triple 
(T',Xi,X2) as follows, 

E[XiT'] = Et[E[XiT'\T]] (162) 

= Et[E[X,\T]E[X,\T]] (163) 

= Var(T') (164) 
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and 



E[X2T']= Et[E[X2T'\T]] (165) 

= Et[E[X2\T]E[Xi\T]] (166) 

and finally, 

E[XiX2] = Et[E[XiX2\T]] (167) 

= Et[E[Xi\T]E[X2\T]] (168) 

where (I168P follows from (]159p . Therefore, the triple {T' , Xi, X2) satisfies 

E[X,T']E[X2T'] 
E[XiX2] ^^-^^7^ (169) 

Now using the fact that 



E[X,X2] = pi2VPiP2 (170) 
E[X,T'] = p,^,,/PJ^ (171) 

E[X2T'] =p2T'/^^ (172) 
and substituting in f ll69p we obtain that the covariance matrix S satisfies 

P12 = PIT'P2T' (173) 

Therefore, from (11511) any jointly Gaussian {Tq,Xig-,X2g) triple with a covariance matrix 
S, with entries (pi2, Pit' 5 P2T') satisfies (H3!l . 

We now arrive at the final step of the evaluation. In particular, we will show that the rates 
of this jointly Gaussian triple (T^, Xig, X2g) will include the rates of the given non-Gaussian 
triple (T, Xi,X2). For the triple (T^, Xig, X2g), we have the following set of inequalities, 

J(Xig;F,1f,|X2g,T;) = /i(r, F^,|X2g, T^) - h{Y,YF,\XrG.X2G,T'a) (174) 

= h{\/ HiqXig + ^/h^XiG + Z2IX2G, Tq) 

-h{Y,Yp,\X,G,X2G,Ta) (175) 

> /i( v^Xi + Z, v^Xi + Z2IX2, T') - h{Y, Yp, |XiG, X2G, Tg) 

(176) 

> /i( v^Xi + Z, xA^Xi + Z2IX2, T', T) - /i(r, F^, |Xi, X2, T) 

(177) 

= /i( v^Xi + Z, v^Xi + Z2IX2, T) - /i(F, F^, |Xi, X2, T) (178) 
= J(Xi;y,FT.|X2,T) (179) 
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where (I176P follows from the fact that (T', Xi, X2) and (T^, Xig, X2g) have the same covari- 
ance matrix S and by using the maximum entropy theorem. Next, fll77p follows from the 
fact that conditioning reduces differential entropy and finally f llTSp follows from the fact that 
T' is a deterministic function of T and by invoking the Markov chain in f ll6ip . Similarly, we 
also have 

I{X2g:Y,Yp,\X^g.T'g) > I{X2;Y,Yp,\X,,T) (180) 
I{X,G,X2G;Y,Yp„Yp,\Ta) > I{X,,X2;Y,Yf„Yf,\T) (181) 

Finally, we have 

I{XiG, X2g; Y) = h{Y) - h{Y\XiG, X2g) (182) 
= hi^/hToX.G + v^X2G + Z)- h{Z) (183) 
> hi^/hToX^ + ^/hioX2 + Z) - h{Z) (184) 
= I{Xi,X2;Y) (185) 

Therefore, we conclude that for any non-Gaussian distribution p{t, xi,X2) G V^g'^^\ there 
exists a jointly Gaussian distribution p{t,Xi,X2) G Vq^ which satisfies the dependence bal- 
ance bound P3|) and yields a set of rates which include the set of rates given by the fixed 
non-Gaussian distribution. Hence, it suffices to consider jointly Gaussian distributions in 
Vg^ to evaluate our outer bound. 

The dependence balance based outer bound can now be written in an explicit form as 
follows, 



VB^i"" = U I (^1' ^2) : ^1 < Jlog (1 + /i(Pit)) 



(PlT,P2T)e[0,l]x[0,l] 



2 

R2 < ^log(l + /2(p2T)) 

Ri + R2< ^log (1 + h{plT, P2t)) 
Ri + R2< ^log (1 + UipiT, P2t)) \ (186) 
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where 



Mpit) = {I - plr)Pi + ¥) (187) 

f2{p2T) = {l-plT)P2(^ + ^) (188) 

/3(pit,P2t) = /i(pir) + /2(P2t) + (1 - P?t)(1 - pIt)PiP2I3 (189) 



. ihioPl + h2oP2 + 2piTp2TVhJl2oPl^) 
f4{plT,p2T) = 2 (190) 

(T 7- 



and 



_ {hi2h2ia% + hohual^ + /iio/j-2io-|J 

~ ^2 „2 ^2 l-'-91J 

The cut-set outer bound given in is evaluated for the Gaussian MAC with user 

cooperation described in (!34 l) -( l36l) as 



C5^c^^= U {(i?i,i?2):i?i<^log(l + (l-pM(^ + ^ 



1, Ox „ /i-^n /i 



^2<^l0g(l + (l-p^)P2(^ + ^ 



A , + ^20^2 + 2pV/lio/l2oPlP2 

i?l + i?2 < -log I 1 H 2 



'Z 



(192) 



We now mention how our outer bound compares with the cut-set bound for the limiting 
cases of cooperation noise variances. 

1. o"|^,cr|^ 0: this case corresponds to total cooperation between transmitters. In this 
case, both dependence balance bound and the cut-set bound degenerate to the total 
cooperation line, 



1, ^ , hoPi + h2oP2 +_2Vhioh2oPiP2 

4 



Rl + R2< ^log 1 + J ^^gg^ 



2. cr^^,o"^2 ~^ ^his '^^^^ corresponds to very noisy cooperation links. In this case, we 
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have 



/i(Pir) 



(1 - plT)hioPi 



(194) 




(1 - f>irV>2oP2 



(195) 



fsiPlT, P2t) 



/i(Pit) +/2(P2t) 



(196) 



< 




(197) 



and the dependence balance bound collapses to the capacity region of the Gaussian 
MAC with no cooperation. On the other hand, the cut-set bound collapses to the 
capacity region of the Gaussian MAG with noiseless feedback [10]. 

Figure 9 illustrates the outer bounds and achievable rate region [2] for the case when 
Pi — P2 — b,a\ — 2 and crl^ = — 1 and hiQ — /i2o = ^12 — h2i — 1. Figure 10 
illustrates the outer bounds for the case when P-^ — P2 — 0"^^ — 1 and (7|^ = crl^ = 20 
and h\Q — /i2o = ^12 = ^21 = 1- For this case, the achievable rate region does not provide 
any visual improvement over no-cooperation. Figure 11 illustrates these bounds and the 
achievable rate region for the asymmetric setting where Pi = P2 = o"| = 1 and (t|^ = af^ = 1 
and hiQ = /i2o = 1, ^12 = 3, /i2i = 2. Figure 12 illustrates these bounds and the achievable 
rate region for the one sided cooperation where Pi = P2 = c"| = 1 and a^^ — crf^ = 1 and 
hio — /i2o — 1) hi2 — 2, /i2i = 0. 
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1.5 




Figure 9: Illustration of bounds for Pi = P2 = 5, (t| = 2, o"|^ = o"|^ = 1 and hio = = 
hi2 = h2i = 1. 
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Figure 10: Illustration of outer bounds for Pi = P2 = = I, o^z^ = <^Z2 — 20 and 
^10 = ^20 = hi2 = /121 = 1- 
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12 Evaluation of VB^fc 

In this section we will explicitly evaluate Theorem 5 for the Gaussian IC with user coop- 
eration described by (jm) - fH7l) in Section 8. We start with step 1 and first characterize the 
set of jointly Gaussian triples (Tg, Xic, X2g) in Vq^ . For this purpose, we rewrite (156!) as 
follows, 

h{YF„YF,\T) + h{YF„YF,\X,, X,, T) < h{YF„YF,\X,, T) + Fp^jXa, T) (198) 

Making use of the following equalities, 

h{YF,,YF,\X^,X2,T) = ^log((27re)V|^4J (199) 
h{YF,,YF,\X^,T) = ilog((27re)4J +/i(FpjXi,T) (200) 
h{YF,,YF,\X2,T) = ilog((27re)4J +/i(Fp,|X2,T) (201) 

we obtain a simplified expression for fll98p as, 

h{YF,,YF,\T) < h{YF,\X,,T) + h{YF,\X2,T) (202) 

which can be further simplified as in the derivation of VB^^'" to the following two equalities, 

J(y^,;Xi|r) = (203) 
I{Yf,-X2\Yf,,T) = Q (204) 

We next follow the same set of arguments used in Section 11 to arrive at the fact that a 
jointly Gaussian triple (T,Xi,X2) satisfies ([2n3])-([2ni iff Xi ^ T ^ X2. 

We can now write the set of rate pairs provided by our outer bound for a jointly Gaussian 
triple {Tg, XiG, X2g) in the set Vg^ as 

Ri<I{X^g.X2g;Yi) (205) 

i?2</(XiG,X2G;l2) (206) 

Ri < I{X,g;Yi,Y2,Yf,\X2g,T) (207) 

R2 < I{X2g;Y,,Y2,Yf,\X,g,T) (208) 

Ri + R2< I{XiG, X2g; Y,, Y2, Yf„ Yf, \T) (209) 

" - ' ^'^^ (210) 



R^ + R2<I{X^G,X2G-.Y^,Y2)} 



where the triple (T^, Xi^, X2g) satisfies the Markov chain Xig Tg X2g- Moreover, 
from the evaluation of step 2 in Section 9, we know that all rate pairs contributed by input 
distributions in Pj^g''"^ are covered by those given in . Therefore, we do not need to 
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consider the set V^q^""^ in evaluating our outer bound. 

We now arrive at step 3 of the evaluation of our outer bound for the Gaussian IC with user 
cooperation. Consider any triple {T,Xi,X2) with a non-Gaussian distribution p(t, xi,X2) G 
'P^Q^^\ with a valid covariance matrix Q. As in the derivation of T>B^q^ , we first construct 
another triple (T',Xi,X2) with a covariance matrix S by selecting 

T' = E[Xi\T] (211) 

Following this step, we next make use of the Markov chain 

T' ^T^ (Xi, X2) ^ (Fi, F2, Yf,,Yf,) (212) 

to show the existence of a jointly Gaussian {T'q,Xig,X2g) with a covariance matrix S and 
which satisfies f lSBl) . 

We now arrive at the final step of the evaluation. In particular, we will show that the rates 
of this jointly Gaussian triple (T^, Xig, X2g) will include the rates of the given non-Gaussian 
triple (T, Xi,X2). For the triple (T(|j, Xig, ^20), we have the following set of inequalities, 

/(Xig; Fi, I2, Yf, IX2G, T'a) = h{Y,, Y2, Yf, \X2g, T'^) - h{Y,, Y2, Yf, \X,g, X2G, Tg) (213) 

= h{X,G + iVl, V^XiG + N2, V^XiG + Z2\X2G, T'^) 

- h{Y,, Y2, Yf, \X,g, X2G, T'c) (214) 



> /i((Xi + Xi, y^Xi + ^2, v^Xi + Z2IX2, r') 

- h{Yr, Y2, Yf, X2G, T'g) (215) 

> /i((Xi + Xi, v^Xi + X2, v^Xi + Z2IX2, t', T) 

- h{Yr, Y2, Yf, |Xig, X2G, T'g) (216) 
= /l((Xi + Xi, v^Xi + X2, v^^i + ^21^2, T) 

-MFi,r2,lFjXi,X2,T) (217) 
= /(Xi;Fi,F2,>F.|X2,T) (218) 

where (I215p follows from the fact that (T', Xi, X2) and (T^, Xic X2g) have the same covari- 
ance matrix S and using the maximum entropy theorem. Next, fl216p follows from the fact 
that conditioning reduces differential entropy and finally fl217p follows from the fact that T' 
is a deterministic function of T and invoking the Markov chain in (12120 . Similarly, we also 
have 

I{X2g]Y^.Y2,Yf,\X^g,T'g)>I{X2;Y^,Y2,YfAX^.T) (219) 
J(XiG, X2g; Vl, Y2, Yf,, Yf,\T'g) > /(Xi, X2; Fi, Y2, Yf,,Yf,\T) (220) 
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Finally, we have 



(221) 
(222) 
(223) 
(224) 



= h{XiG + VhX2G + A^i) - KNi) 
> h{Xi + VbX2 + iVi) - h{N^) 
= /(Xi,X2;ri) 



and similarly, we also have. 



I{X,G,X2G;Y2)>IiX,,X2;Y2) 
I{X,g,X2g;YuY2) > I{X,,X2;Y,,Y2) 



(225) 
(226) 



Therefore, we conclude that for any non-Gaussian distribution p{t, xi,X2) G Vp^G ^ 
there exists a jointly Gaussian distribution p{t, Xi, X2) G Vq^ which satisfies the dependence 
balance bound fl56|) and yields a set of rates which includes the set of rates given by the 
fixed non-Gaussian distribution. Hence, it suffices to consider jointly Gaussian distributions 
in Vg^ to evaluate our outer bound. 

The dependence balance based outer bound can now be written in an explicit form as. 



VB, 



•ic _ 
uc — 




R2 < ^^Og(l + /2(PiT,P2t)) 
i?l<^log(l + /3(piT)) 
P2<^l0g(l + /4(P2T)) 
Rl+R2< ^log (1 + /5(P1T, P2t)) 




(227) 



where 



/i(Pit,P2t) 



(Pi + 6P2 + 2piTP2TVwy^) 



(228) 



/2(PlT, P2t) 




(229) 




(230) 
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MP2t) = (1 - pIt)P2 ( + + ^ ) (231) 

'^Ni "N2 ^Zi 



J5{P1T,P2T> = h[PlT,P2T> + j2[PlT,p2T) H 2 2 l^Ji) 

Mpit,P2t) = Mpit) + /4(P2t) + (1 - P?t)(1 - pIt)PiP2P (233) 

where 

^ ^ ^12^21 ^ (1 - ^MY ^!^fj_^ ^234) 

The cut-set outer bound given in (l5l)- (fT0l) is evaluated for the Gaussian IC with user 
cooperation described in (lHll - lHTI) as 



CS 



uc= U |(^i,^2):i?i<^log(l + ^ 



R2 < -log I 1 + 



fi,<ilog(l + (l-/)P.(;^ + -^ + ^ 
fl.<ilog(l + (l-/)P.(;^ + ;^ + ^ 



(235) 



where 



Pi + &P2 + 2pv/6iy^ 
^i(p) = 2 (236) 



aPi + P2 + 2pvW^ 
^i(p) = 2 (237) 

Figure 13 illustrates our outer bound, cut-set bound, an achievable rate region with 
cooperation [7], capacity region without cooperation [18] for the case when Pi = P2 = 
(t|^^ = 0"!^^ = 1 and cr|^ = o"!^ = 1 and a = b = 1 and hi2 = /i2i = 2. Figure 14 
illustrates the outer bound, cut-set bound and achievable region without cooperation [9] 
when Pi = P2 = a%_^ = a%_^ = 1 and cr|^ = a^^ = 1 and a = b = 0.5 and hu = /i2i = 0.1. 
Figure 15 illustrates our sum rate upper bound and the cut-set bound as function of h, where 
h = hi2 = h2i and Pi = P2 = 0"|y^ = = 1 and = crf^ = 1, a = b = 0.5. 
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Figure 13: Illustration of bounds for Pi = P2 = o"|y^ = aj^^ = 1, cr|^ = = 1 and a = b = 1 
and hi2 = = 2. 
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Figure 14: Illustration of bounds for for Pi = P2 = cr^^ = cr^^ ~ 1' '^Zi — '^'z-z ~ ^ ^"^^ 
a = b = 0.5 and hi2 = /i2i = 0.1. 
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Figure 15: Illustration of sum-rate upper bound and the cut-set bound as a function of h, 
where h = hi2 = /i2i- 

13 Conclusions 

We obtained new outer bounds for the capacity regions of the two-user MAC with generalized 
feedback and the two-user IC with generalized feedback. We explicitly evaluated these outer 
bounds for three channel models. In particular, we evaluated our outer bounds for the 
Gaussian MAC with different noisy feedback signals at the transmitters, the Gaussian MAC 
with user cooperation and the Gaussian IC with user cooperation. Our outer bounds strictly 
improve upon the cut-set bound for all three channel models. 

For the evaluation of our outer bounds for the Gaussian scenarios of interest, we proposed 
a systematic approach to deal with capacity bounds involving auxiliary random variables. 
This approach was appropriately tailored according to the channel model in consideration 
which permitted us to obtain explicit expressions for our outer bounds. To evaluate our 
outer bounds, we have to consider all input distributions satisfying the dependence balance 
constraint. The main difficulty in evaluating our outer bounds arises from the fact that 
there might exist some non-Gaussian input distribution p{t, Xi, X2) with a covariance matrix 
Q, such that p{t, xi,X2) satisfies the dependence balance constraint but there does not exist 
a jointly Gaussian triple with the covariance matrix Q satisfying the dependence balance 
constraint. Therefore, the regular methodology of evaluating outer bounds, i.e., the approach 
of applying maximum entropy theorem [17] fails beyond this particular point. Through our 
explicit evaluation for all three channel models, we were able to show the existence of a 
jointly Gaussian triple with a covariance matrix S which satisfies the dependence balance 
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constraint and yields larger rates than the fixed non-Gaussian distribution. 

In particular, for the case of Gaussian MAC with noisy feedback, we made use of a 
recently discovered multivariate EPI [13], which is a generalization of Costa's EPI [14]. It is 
worth nothing that this result could not be obtained from the classical vector EPI. For the 
case of Gaussian MAC with user cooperation and the Gaussian IC with user cooperation, 
our proof closely follows a recent result of Bross, Wigger and Lapidoth [15] and [16] for the 
Gaussian MAC with conferencing encoders. 

14 Appendix 

14.1 Proof of Theorem 1 

We will prove Theorem 1 by first deriving an upper bound for Ri as 

nRi = H{Wi) = H{Wi\W2) (238) 
= IiW,;Y'',Yg\W2) + HiWilW^^Y^'^Y^^) (239) 

< I{Wi; F", Y^JW2) + neS"^ (240) 

n 

= ^ I{W^■, Y,, YfJW2, Y'-\ Y^;') + ne^r^ (241) 

i=l 
n 

= J2{H{Y„ Yf,,\W2, Y^-\ Y^;') - H{Y,, Yf,,\W,, W2, Y'-\ Y],;')) + ne'C^ (242) 

i=l 
n 

= Y,iH{Yi, YfJW2, Y'-\ y;;i) - H{Y„ YfJX2., Wu W2, Y'-\ Y^;')) + nej") 

i=l 

(243) 

n 

< J2(H{Y^, YfJX2,, Y^;') - H{Y,, Yp,,\X2., W,, W2, Y'-\ Y^;')) + nej") (244) 

i=l 

n 



< J2iH{Y„ YF,,\X2i, Y^;') - H{Yi, YfJX^,, W,, W2, Y'-\ Y^;')) + ne^"^ (245) 

i=l 

n 

= J2iH{Y,,YFjX2^,Y;r') - H{Y„YpjXu,X2„Y;r')) +ne^r^ (246) 

i=l 
n 

= ^ I{Xu; Y,, YfJX2^, Y^;^) + nei"^ (247) 



i=l 



= nI{X,Q- Yq, Yf,q\X2q, Q, Y^-') + nel") (248) 
= nJ(Xi; Y, Yf, IX2, T2) + nej"^ (249) 

where fl240p follows from Fano's inequality [17], fl243p follows from the fact that X2i is a 
function of {W2,Yp~'^) and by introducing X2i in both terms, (1244^ follows from the fact 
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that conditioning reduces entropy and we drop {W2, Y^~^) from the conditioning in the first 
term, (12451) follows from the fact that conditioning reduces entropy and by introducing Xn 
in the second term and (12461) follows from the memoryless property of the channel. Finally, 
we define Xi = Xig, X2 = X^q, T, = (Q, y^"'), T2 = (Q, Y^-'), Y = Yq, Yp, = Yp,Q and 
Yp2 = Yp^Q, where Q is a random variable which is uniformly distributed over {1, . . . ,n} and 
is independent of all other random variables. Similarly, we have 

R2 < I{X2;Y,Yp,\X^,T,) (250) 
Ri + R2< /(Xi, X2; Y, Yp,,Yp, \TuT2) (251) 

In addition to (12511) . we also have the following sum-rate constraint which also appears in 
the cut-set outer bound, 

n{Ri + R2) = H{W,,W2) (252) 
= I{Wi,W2;Y'') + H{Wi,W2\Y'') (253) 

< 1(1^1, 1^2; V-") +ne(") (254) 

n 

= J2iH{Yi\Y'"^) - H{Yi\Wi, W2, Y'-^)) + ne^""^ (255) 
1=1 

n 

< Y,{H{Y,\Y'-^) - H{Y,\Xu, Wu W2, Y'~')) + ne^ (256) 

i=l 
n 

= Y^iHiny''') - H{Y,\Xu,X2^)) + ne'^^^ (257) 

i=l 

n 

< Y,{H{Yi) - H{Y,\Xu,X2^)) + ne(") (258) 

i=l 
n 

= 5^/(XH,X2.;r.) + ne(") (259) 

i=l 

= nI{X^Q,X2Q;YQ\Q)+ne^^^ (260) 

<nI{X^Q,X2Q■,YQ)+ne^^^ (261) 

= n/(Xi,X2;F)+ne(") (262) 

It is necessary to include this seemingly trivial upper bound on the sum-rate. The reason for 
including this sum-rate upper bound is that one cannot claim that for any input distribution 
p(ti,t2,Xi,X2), we have J(Xi, X2; Y, Yp^, YpjlTi, T2) < I{Xi, X2;Y). In other words, we 
cannot claim that the sum-rate bound in (I262p will always be redundant. Therefore, by 
including it, we can make sure that our outer bound is at most equal to the cut-set outer 
bound but never larger than it. Although, as we will see in the proof of Theorem 3 for the 
case of noisy feedback, the sum-rate upper bound in (I262p will turn out to be redundant. 
The proof of the dependence balance constraint in f|T6l) is along the same lines as in [1] 
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by starting from the inequality 

< I{W,; W2\Y^^, F^f ) - I{W,; W2) 

to arrive at 

IiX,;X2\T,,T2) < I{X^-X2\Yp,,Yf,,T^,T2) 
This completes the proof of Theorem 1. 

14.2 Proof of Theorem 2 

We will prove Theorem 2 by first deriving an upper bound for Ri as 

nRi^H{Wi) = H{Wi\W2) 

= IiW^■, Fi", 1^, Yg\W2) + HiW,\W2, Y^ , Y^, Yg) 
<IiW,;Y{^,Y,^,Y^jW2)+ne^;'^ 

n 

= J2 nWi; Yu, Y2i, YfJW2, Yt\ Yt\ Y^;') + ne'C^ 
1=1 

n 



= J2(H{Yu, Y2i, YF,i\W2, Yt\ Yt\ Y^;') 

i=l 

- H(Yu, Y2i, YF,i\W,, W2, Yr\Yt\ Y^;')) + neS^^ 

n 

= J2iH{Yu, Y2^, YF,i\W2, X2i, Yr\ Yl\ Y^;') 

i=l 

- H{Yu, Y2i, YFjX2i, Wu W2, Yl-\ Yi-\ Y^')) + nej") 

n 

< Y2i, Yp2i\X2i, Yp^ ^) 

1=1 

- H{Yu, Y2^, YF,i\X2i, W,, W2, Yr\ Y;-\ Y^;')) + ne^^^ 

n 

<Y,{H{Yu.Y2i,YF,i\X2i,Y^p-^) 

1=1 

- H{Yu, Y2i, YfJXu, X2i, Wi, W2, Yf, Yl\ Y^r')) + neS^^ 

n 

) Y2i, Yp^i \X2i,Y^;') - H{Y,i, Y2i, Yp^ilXii, X2i, Yp^ 

')) + 

1=1 

n 

'1 Yii, Y2i, Yp^i \X2^,Yp-^) + neT' 

1=1 

= n/(Xi; Fi, F2, Yf, 1^2, T2) + ne^'^ 



43 



where (1267P follows from Fano's inequality [17], fl270p follows from the fact that is a 
function of {W2,Yp~'^) and by introducing in both terms, fl27ip follows from the fact 
that conditioning reduces entropy and we drop {W2, Y^"^, 1^2*"^) from the conditioning in the 
first term, (12721) follows from the fact that conditioning reduces entropy and by introducing 
Xii in the conditioning in the second term and (I273P follows from the memoryless property 
of the channel. Finally, we define X, = X,q, X2 = X2Q, T, = (Q,F^-'), T2 = (Q,F|-^), 
Yi = YiQ, Y2 = Y2Q, Yfi = Yp-^Q and Yp^ = Yp^g, where Q is a random variable which is 
uniformly distributed over {1, ... ,72} and is independent of all other random variables. 
Similarly, we have 

R2<I{X2;YuY2,Yf,\Xi,Ti) (276) 
Ri + R2< /(Xi, X2; Fi, Y2, y>„ Yf,\Tu T2) (277) 

and we also have from the cut-set bound 

i?i </(Xi,X2;yi) (278) 
R2<I{X2,X2;Y2) (279) 
/?i + i?2</(Xi,X2;yi,y2) (280) 

The proof of the dependence balance constraint is along the same lines as in [1] by starting 
from the inequality 

< /(H^i; W2\Y^,,Y^;) - I{W^- W2) (281) 

to arrive at 

/(Xi;X2|ri,T2) </(Xi;X2|yi.„yi.„Ti,T2) (282) 
This completes the proof of Theorem 2. 

14.3 Proof of Theorem 3 

For any MAC-GF, with transition probabilities in the form of (!28|) . we will obtain a strength- 
ened version of Theorem 1. We start by obtaining an upper bound on Ri as 

ni?i = H{Wi) = H{Wi\W2) (283) 
= I{Wv, r", Y^^, Yg\W2) + H{Wi\W2, F", Y^^,Yg) (284) 
< liW,; y", Y^^,Y^JW2) + ne!") (285) 

n 

= J2 HWi, y^, yF,^, YfJW2, Y'-\ YI,;\ y-^) + neS") (286) 
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J2 HWi; nW2, Y'-\ Yp;\ + nej") (287) 

i=l 

n 

Y,{H{Y,\W2. Yp;\ - 1^2, Y^-\ Yp;\ + neS") (288) 



= 5^(if(r,|X2„ 1^2, >^;r\ Y^f^') - H{Y,\X2,, W,, W2, Y'-\ Y^~\ Y^;')) + nef^ 

i=l 

(289) 

n 

< Y,iH{nx2i, W2, Y'-\ Yl,;\ Y^;') - H{Yi\Xu, w,, W2, y'-\ y^;')) + nel"^ 

i=l 

(290) 

n 

= J2iH{nx2., W2, Y^-\ r;;i) - i/(y,|Xi„ x^,, y^;\ y^;')) + nef~^ (291) 



i=l 
n 



< Y,iH{nX2u Y^;') - H{Y,\Xu. X2,, F;;^)) + neS"^ (292) 

i=l 
n 

= 5^ /(Xi,; r,|X2„ f;;1) + neS") (293) 

i=l 

= nI{X,Q- Yq\X2q, Q, Y^~\ Yg-^) + neS") (294) 
= n/(Xi;F|X2,T) + nel"^ (295) 

where (12851) follows from Fano's inequality [17], and (12870 follows from the following Markov 
chain, 

(y^,„ Yf,,) ^ ^ (1^1, 1^2, v-^-S v;;') (296) 

and (I289p follows from the fact that X2i is a function of (1^2, Y'p'^), (l290D follows from the fact 
that conditioning reduces entropy, (12911) from the memoryless property of the channel and 
(]292p follows by dropping (PF2, Y'''^) from the first term and obtaining an upper bound. We 
finally arrive at (12951) by defining the auxiliary random variable T = (Q, Yp^^ , Yp~^), where 
Q is a random variable which is uniformly distributed over {1, . . . , n} and is independent of 
all other random variables. Similarly, we also have 

i?2</(X2;F|Xi,T) (297) 

and 

i?i + i?2 < /(Xi, X2; r, Yp,,Yp,\T) (298) 

= /(Xi,X2;r|T) (299) 
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where (12990 follows from the Markov chain (Yf^jYf^) ^ Y ^ {Xi, X2,T). Moreover, as a 
consequence of (12991) . the sum-rate bound 

Rl + R2<I{X^,X2■,Y) (300) 

obtained in (12621) is redundant for any MAC-GF with transition probabilities in the form of 
(1281) . The proof of the dependence balance constraint is the same as in Theorem 1. This 
completes the proof of Theorem 3. 



14.4 Proof of Theorem 4 

The main idea behind the strengthening of Theorem 1 for user cooperation is to use the 
special conditional probability structure of (1371) . Using this conditional structure, we will 
obtain an outer bound involving only one auxiliary random variable. We first note that 
without any loss of generality, the conditional distributions p{yFii\x2i) and p{yF2i\^ii) can be 
alternatively expressed as two deterministic functions [19], [20], i.e., 

YF,^ = gi{X2^, Zu) (301) 
YF,, = g2iXu,Z2,) (302) 

where the random variables Zu and Z2i are independent and identically distributed for all 
2 G {1, . . . , n} and are also independent of the messages (Wi, W2). We now prove Theorem 
4 by first obtaining an upper bound on R\ as follows, 

nRi = H{Wi) = H{Wi\W2) (303) 

= liW,; F", , Z^\W2) + H{Wi\W2, F", Y^^, Z^) (304) 

< I{Wr, F", , Z'^\W2) + ne'C^ (305) 

= liWi; Z^\W2) + HWi, F", Y^JW2, Z^) + ne^") (306) 

= I{Wi; F^ Yg\W2, Z^) + neS") (307) 

n 

= I^Wi- F„ YF,^\W2, Y'-\ Y^;\ + neS") (308) 

i=l 

n 

An) 



J2iH{Y., YF,^\W2, F-\ F-\ ZD - H{Y,, Yf,.\Wu W2, Y'-\ Y^;\ Z^)) + ne^ 

i=l 

(309) 

n 

J2iH{Y,, yfJW2, xr \ zr, Y'-\ f;-i) 



i=l 



H{Yi, YF,i\W,, W2, Y'~\ Y^;\ Z^)) + neS") (310) 
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1=1 

- H{Y,, YfJXu, X2^, Y'p-\ Y^;\ Wi, W2, Y'-\ Z^)) + ne^^ (311) 

n 
i=l 

- H{Y„ YpjX,,, Y^;\ r-i)) + neS") (312) 

n 

= J2iH{Y,, YfJX2., xif\ Y^;\ Y^;\ Y^-\ W2, ZD 

- H{Yi, Yf,,\Xu, Y'f^\ r;;i)) + neS"^ (313) 
< J](i/(F„ Yp,i\X2i, Yp;\ r;;i) - HiXi, Yf,,\Xu, X2„ r;;^)) + ne^"^ (314) 



n 



i=l 



= HXu; Y„ Fi.,.|X2„ F;;^) + mS") (315) 
j=l 

= n/(XiQ; Yq, Yf,q\X2q, Q, Y^-\ Yg'') + nej") (316) 

= nJ(Xi; F, y^^.lXs, T) + ne^") (317) 

where (I305P follows from Fano's inequality [17], (13071) follows from the independence of 
(1^1,1^2) and Z", fl310p follows by adding {X2i., X^'^) in the conditioning of the first term. 
This is possible since {X2i^ X""^^) is a function of {W2,Ylr^). We further upper bound by 
introducing (Xij, X2i, Y^"^) in the conditioning in the second term to arrive at (13111) . In 
(13121) . we use the memoryless property of the channel to drop {Wi,W2,Y^~^ , Z^) from the 
conditioning in the second term while retaining (Y^~^, Y^~^). 

Next, we make use of the special channel structure of fl37|) . More specifically, using 
(13011) . we observe that Yp~^ is a deterministic function of ^2"^ and Zl~^ and therefore, 
it is introduced in the conditioning in the first term in (I313p . This is the crucial part of 
the proof which enables us to obtain an outer bound involving only one auxiliary random 
variable as opposed to two auxiliary random variables. Next, we upper bound (I313P by 
dropping {W2,Y'^~^ , X2''^ , Z^) from the first term to arrive at (1314^ . Finally, we define 
T = {Q,Yg'\Yg-'), Xi = XiQ, X2 = X2Q, Y = Yq, Yp, = Yf,q and Yp, = Yf,q, where 
Q is a random variable which is uniformly distributed over {1, . . . ,n} and is independent of 
all other random variables. Similarly, we have 

R2<I{X2;Y,Yf,\X^,T) (318) 
Ri + R2 < I{X,,X2;Y,Yf,,Yf,\T) (319) 



The derivation of the constraint ( 14T|) is the same as in Theorem 1 and is omitted. Moreover, 
from the proof of the dependence balance constraint in (fT6|) . we observe that Y^~^ and 
Yp~^ appear together in the conditioning. Therefore, from our earlier definition of T = 
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{Q, Yp^ ^, we directly have from the proof of (fT6|) 

J(Xi;X2|T) < I{X^,X2\Yf„Yf„T) (320) 
This completes the proof of Theorem 4. 

14.5 Proof of Theorem 5 

The idea behind obtaining a strengthened version of Theorem 2 for IC with user cooperation 
is to use the special transition probability structure of (HHI) . Using the same argument as in 
the proof of Theorem 4, we can express Yp^j and Yp^i as, 

Yf,, = g,{X2., Zu) (321) 
Yp,, = g2iXu,Z2,) (322) 

where the random variables Zu and Z2i are independent and identically distributed for all 
z G {1, . . . ,n} and are also independent of the messages {Wi, W2)- We now prove Theorem 
5 by first obtaining an upper bound on Ri as follows, 

nRi = H{Wi) = H{Wi\W2) (323) 

(324) 
(325) 
(326) 
(327) 



= H{Wi) = H{Wi\W2) 

= I{W,; Y,\ Y^, Y^^,Zl\W2) + H{Wr\W2, Y^ , Y^ , F/f , 
< /(W^i; n", , ZrilVa) + neS"^ 

= /(H^i; Zl\W2) + /(W^i; K,", F^JW^2, + nej") 
= J(l^i; n", y;f 11^2, ^r) + ne^? 

n 



n 

= J2 HWi; Fi„ Y2,, YfJW2, Yr\ Yr\ Y^;\ Z^) + ne^") 

i=l 
n 

= ^(if(Yi„ 1^2., YF,i\W2, Yr\ Y^-\ Y^;\ Z^) 

i=l 

- H{Yu, Y2^, YfJWi, W2, YI-\ Y^-\ Y^;\ Z"^)) + neS^^ 

n 

= ''^iH{Yii,Y2i,YF2i\W2,X2i,X2 ^,Zl,Yl \ ^'^F2^) 

i=l 

- H{Yu, Y2,, Yf,,\W^, W2, Yl-\ Yl~\ Y^;\ Z^)) + nef^ 

n 
i=l 

- H{Yu, ^2^, YFadXi,, X2„ Y'p-\ Y^;\ Wi, W2, YI-\ Y^-\ Z^)) + nef^ 



(328) 



(329) 



(330) 



(331) 
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1=1 

- H{Yu, Y2^, YfJXu, y;;^)) + nel") (332) 

n 
i=l 

- H{Yu, Y2,, YpjXu, + neS") (333) 

n 

< ^^{H (Yii, Y2i, Yp^i\X2i, Yp^ Yp^ "*") — H{Yii, Y2i, Yp^i\Xii, X2i, Yp_^ ^, Yp^ ^)) + ne["^ 

(334) 

n 

= -^(-^li! ^li, Y2i, Yp^i\X2i, Yp^ Yj^^ ^) + ne^^ (335) 

= nI{X,Q- Y,Q, Y2Q, Yp,q\X2q, Q, Y^-\ Y^-') + ne^^ (336) 
= n/(Xi; Fi, F2, Yp^ \X2, T) + ne[") (337) 

where fl325p follows from Fano's inequality [17], (13271) follows from the independence of 
(Vri,PF2) and Z", fl330p follows by adding {X2i,Xlf^) in the conditioning of the first term. 
This is possible since {X2i, X'!^^) is a function of {W2,Yl^^). We further upper bound by 
introducing (Xij, X2i, Y^~^) in the conditioning in the second term to arrive at (13311) . In 
(13321) . we use the memoryless property of the channel to drop (VTi, 14^2, ^2~^5 ^i) from 
the conditioning in the second term while retaining (y^~^,y^~^). 

Next, we make use of the special channel structure of fHHj) . More specifically, using 
(13211) . we observe that Y^^^ is a deterministic function of Xlf^ and Z]^^ and therefore, it 
is introduced in the conditioning in the first term in (I333p . Next, we upper bound (13330 
by dropping (VF2, -^2~^ ^2~^ ^1") fro^^ the first term to arrive at (I334p . Finally, we 
define T = (Q, F^^"^), X, = X,q, X2 = X2Q, Y, = Y,q, Y2 = Y2Q, Yp, = Yp,Q and 

Yp^ = Yp^Q, where Q is a random variable which is uniformly distributed over {1, . . . ,n} and 
is independent of all other random variables. Similarly, we have 

i?2 </(X2;Fi,F2,1fJXi,T) (338) 

The derivations of the remaining constraints are similar to the proof of Theorem 2 since 
both Y^-^ and Y^;^ appear together in the conditioning and T can be defined appropriately 
without any difficulty. The proof of dependence balance constraint in ( 156|) is the same as in 
Theorem 2. This completes the proof of Theorem 5. 
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14.6 Proof of (ESD 

In the following derivation of (ffSl) . we have dropped conditioning on T = t, for the purpose 
of simphcity. Substituting (ED!), (EI]), (El) and (JMl) in ([76]), we have 

NiA^/^Y + \)=N{^Y + Vi,V2) (339) 
1 ^hiV-^Y+v„v,) 



(27re) 



1 gMv^y+Vi) (341) 



We also note the following inequahty, 



h{^Y + VI) > ^log(e2'^(^^) + 27re) (342) 
= ^log(/€e2'^(^) + 27re) (343) 

where (I342p follows from the scalar EPI [17] and (13431) follows from the fact that for any 
scalar c, h{cY) = h{Y) + \og{\c\) [17]. Substituting fl343D in flMTll . we obtain 



Similarly, we also have 



^,A..Y.V,,(-^^!!^)" ,345) 



Therefore, we have 



' , 2h{Y) I 9 \ ^/"^ 

/iiV(A;/2Y + V) + (1 - /i)iV(Af Y + V) > ( ^ \ ] (346) 



(27re) 



Moreover, the right hand side of (!76|l simplifies to. 



NUfiA, + (1 - /i)A2)'/'Y + V) = ^_ehiVJ^+Vi,y/(T^Y+V2) (347) 

(27re) 

(27re) ^ ^ 

= ^ pMyF,,i>,) (349) 

(2vre) 
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Using 03450 - fl349p and substituting in (1761) . we obtain 




:^^(^) + 27re 
(27re) 



) 



1/2 



1 



< 



(350) 



Simplifying (I350p by substituting the value of k and reintroducing the conditioning on T = t, 
we have the proof of (|751) . 
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